In [1]:
#Importing the required libraries
import os
import pandas as pd
pd.core.common.is_list_like = pd.api.types.is_list_like
import pandas_datareader.data as web
import matplotlib.pyplot as plt
%matplotlib inline
import numpy as np
from nsetools import Nse
from nsepy import get_history
import seaborn as sns
import plotly.express as px
from datetime import datetime, timedelta, date
1. Loading Data¶
The list of stocks of different indices have be taken from 'nseindia.com'
The data for 'P/E Ratio' is taken from 'valueresearchonline.com'
In [2]:
#Loading from excels
Bluechip = pd.read_excel('Data\\NIFTY50_PE_Ratio_19_Sept_2019.xlsx')
Midcap= pd.read_excel('Data\\Midcap100_PE_Ratio_19_Sept_2019.xlsx')
Smallcap = pd.read_excel('Data\\SmallCap100_PE_Ratio_19_Sept_2019.xlsx')
In [3]:
#Creating a new column identifying the stock type
Bluechip['Category'] = "Bluechip"
Midcap['Category'] = 'Midcap'
Smallcap['Category'] = 'Smallcap'
Stock Selection Strategy¶
2. Selecting stocks with PE values ranging between 4-15 from different sectors.¶
In [4]:
#Selecting stocks based on their index numbers
Bluechip_stocks = Bluechip[['Company', 'Symbol', 'Price', 'PE', 'Category']].iloc[[41, 24, 33, 18]]
Midcap_stocks = Midcap[['Company', 'Symbol', 'Price', 'PE', 'Category']].iloc[[42, 57, 25, 30]]
Smallcap_stocks = Smallcap[['Company', 'Symbol', 'Price', 'PE', 'Category']].iloc[[29, 7, 96, 19]]
In [5]:
#Merging all the 3 dataframes into 1
portfolio_stocks_list = pd.merge(Bluechip_stocks, Midcap_stocks, how = 'outer')
portfolio_stocks_list = pd.merge(portfolio_stocks_list, Smallcap_stocks, how = 'outer')
print(portfolio_stocks_list.shape)
In [6]:
#Final list of stock for portfolio
portfolio_stocks_list
Out[6]:
- The final list of stocks have been selected manually 4 from each cap category with a diverse P/E ratio
- One from each sector
- All stocks have low P/E value ranging from 1.8 to 20 to see how the weight allocation change among them
3. Downloading data for the Selected stocks in 'portfolio_stocks_list'¶
- The data is downloaded using the 'NSE' python library.
In [7]:
#nse = Nse()
#Data Download
start = date(2016,6,6)
end = date(2019,7,2)
portfolio_data = []
for i in portfolio_stocks_list.Symbol.values:
portfolio_data.append(get_history(i, start = start, end = end))
portfolio_stocks_data = pd.concat(portfolio_data)
In [8]:
print(portfolio_stocks_data.shape)
portfolio_stocks_data.head()
Out[8]:
In [9]:
#Verifying whether all the selected stocks have data for the specified period
portfolio_stocks_data['Symbol'].value_counts()
Out[9]:
In [10]:
#Subsetting the required fields
portfolio_stocks_df = portfolio_stocks_data[['Symbol', 'Prev Close']]
In [11]:
#Renaming column to avoid spaces in the column header
portfolio_stocks_df.rename({'Prev Close' : 'Prev_close'},inplace = True)
In [12]:
#Using pivot to create stocks as columns
portfolio_stocks_df = portfolio_stocks_df.pivot(columns= 'Symbol', index= portfolio_stocks_data.index)
#Setting the columns
portfolio_stocks_df.columns = [col[1] for col in portfolio_stocks_df.columns]
#Filling 'NaN' if any
portfolio_stocks_df.fillna(0, inplace = True)
In [13]:
portfolio_stocks_df.head()
Out[13]:
Now that we have clean data for 12 stocks from 3 different categories let's see the price movement for each stock
3. Data Exploration¶
3.1 Plotting the prices of the stocks¶
In [14]:
from pandas.plotting import register_matplotlib_converters
register_matplotlib_converters()
In [15]:
plt.figure(figsize= (14, 8))
for c in portfolio_stocks_df.columns:
plt.plot(portfolio_stocks_df.index, portfolio_stocks_df[c], label = c, lw = 3)
plt.legend(loc = 'best', fontsize = 12)
plt.ylabel('Price in INR')
Out[15]:
It looks like HeroMotoCo and Venkeys have a relatively high which makes to difficult to understand the trend of other stocks.
3.2. To understand better about the volatility of the stocks we will be plotting them using the daily returns / percentage change¶
In [16]:
#Calculating the percentage change for each stock
returns = portfolio_stocks_df.pct_change().dropna()
In [17]:
pd.set_option('display.max_rows', 800)
In [18]:
#Creating returns for each category to analyse the volatility easily
returns_bluechip = returns[['TATASTEEL', 'IBULHSGFIN', 'NTPC', 'HEROMOTOCO' ]]
returns_midcap = returns[['HEXAWARE', 'LICHSGFIN', 'EDELWEISS', 'FEDERALBNK' ]]
returns_smallcap = returns[['FLFL', 'BAJAJELEC', 'VENKEYS', 'CYIENT' ]]
In [19]:
#1. Plotting Bluechip Companies returns
plt.figure(figsize = (14, 8))
for c in returns_bluechip.columns.values:
plt.plot(returns_bluechip.index, returns_bluechip[c], lw = 2, label = c)
plt.legend(loc = 'upper right', fontsize = 12)
plt.ylabel('Percentage Change')
plt.title('Bluechip Stocks Returns Anaysis')
#2. Plotting Midcap Companies returns
plt.figure(figsize = (14, 8))
for c in returns_midcap.columns.values:
plt.plot(returns_midcap.index, returns_midcap[c], lw = 2, label = c)
plt.legend(loc = 'upper right', fontsize = 12)
plt.ylabel('Percentage Change')
plt.title('Midcap Stocks Returns Anaysis')
#3. Plotting Smallcap Companies returns
plt.figure(figsize = (14, 8))
for c in returns_smallcap.columns.values:
plt.plot(returns_smallcap.index, returns_smallcap[c], lw = 2, label = c)
plt.legend(loc = 'upper right', fontsize = 12)
plt.ylabel('Percentage Change')
plt.title('Smallcap Stocks Returns Anaysis')
Out[19]:
From the above three graphs we can clearly see that Blue chip stocks have low standard deviation whereas the standard devialtion increases for Midcap and higher for Small cal stocks. So, at this point we can assume that if we make a high risk - high return portfolio the weights allocation could be higher for small cap stocks.
4. Model Building¶
4.1. Monte Carlo Simulation¶
- My basic approach to building a Portfolio Optimization is based on the Modern Portfolio Theory by Harry Markowitz which states that if an investor is willing to take higher risk then it needs to be compensated with high return. If a portfolio gives same return for a given risk then the investor will choose the one with low risk.
- Porfolio performance is calculated using Sharpe Ratio. (Expected Returns - Risk free rate) / Portfolio standard deviation
- Efficient Frontier gives us the marks the set of optimal portfolios that offer the highest expected return for a defined level of risk or the lowest risk for a given level of expected return.
4.1.1. Calculating¶
- Annual returns
- Annual volatility / risk
- Creating a covariance matrix to calculate portfolio volatility / risk
In [20]:
#Returns is calculated by percentage change of each stock
returns = portfolio_stocks_df.pct_change().dropna()
mean_returns = returns.mean()
In [21]:
mean_returns
Out[21]:
In [22]:
#Creating the covariance matrix of returns
cov_matrix = returns.cov()
4.1.2 Portfolio Perfomance¶
To assess a portfolio's performance I'll be using the basic Sharpe Ratio.
Sharpe Ratio can be calculated using the formula:
(Expected return - Risk free rate) / Portfolio standard deviation
The 'annual_portfolio_performance' function is calculating the return and risk for a given weights, mean_returns and covariance matrix.¶
In [23]:
def annual_portfolio_performance(weights, mean_returns, cov_matrix):
annual_returns = np.sum(mean_returns * weights) * 250
annual_risk = np.sqrt(np.dot(weights.T, np.dot(cov_matrix, weights))) * np.sqrt(250)
return annual_returns, annual_risk
The function 'random_portfolios' 'is¶
- Generating a random set of weights for given number of portfolios and assets
- Calculating the portfolio standard deviation (risk), return and Sharpe ratio for a given set of weights
In [24]:
def random_portfolios(num_assets, num_portfolios, mean_returns, cov_matrix, risk_free_rate):
#creating a 3d array with length specied in 'num_portfolios'
results = np.zeros((3,num_portfolios))
#Generate random weights and create a list of all the random generated weights
weights_record = []
#For 'n' number of portfolios we are generating random weights
for i in range(num_portfolios):
weights = np.random.random(num_assets)
weights /= np.sum(weights)
#'weights_record' is the list of weights for a given number of portfolios and assets
weights_record.append(weights)
#Using the above created 'annual_portfolio_perfomance' function we generate portfolio standard deviation and returns
portfolio_return, portfolio_std_dev = annual_portfolio_performance(weights, mean_returns, cov_matrix)
# So for every set of weights we are calculating the risk, return and sharpe ration and saving it in results.
results[0,i] = portfolio_return
results[1,i] = portfolio_std_dev
results[2,i] = (portfolio_return - risk_free_rate) / portfolio_std_dev
return results, weights_record
In [25]:
#Defining the required values for the function 'random_portfolios' input
num_assets = portfolio_stocks_df.shape[1]
num_portfolios = 25000
# The risk free rate is taken from the govt bond annual return rate.
risk_free_rate = 0.069
Using the function 'random_portfolios' now we have 25000 randomly generated weights for 4 assets and their respective returns, portolio standard deviation (risk) and Sharpe ratio.
1.3 Visualising the results of Maximum Sharpe Ratio and min volatility / risk¶
- Getting weight, portfolio returns, risk and Sharpe Ratio using the above 'random_portfolio' function
- Creating a dataframe of weights whose Sharpe ratio is highest
- Creating a dataframe of weights with low volatility / risk
- Printing all the results of Sharpe ratio and miminum risk returns
- Visualizing the returns, risk and sharpe ratio
In [50]:
def efficient_frontier_random_portfolios(num_assets, num_portfolios, mean_returns, cov_matrix, risk_free_rate):
#Step 1
final_results, final_weights = random_portfolios(num_assets, num_portfolios,mean_returns, cov_matrix, risk_free_rate)
#Step 2
#Taking the maximum values of 'results[2]' which is the sharpe ratio
max_sharpe_indexId = np.argmax(final_results[2])
#Getting the returns and risk for max sharpe ratio
portfolio_return, portfolio_volatility = final_results[0, max_sharpe_indexId], final_results[1, max_sharpe_indexId]
#Creating a dataframe of the final weights of max sharpe ratio
max_sharpe_allocation = pd.DataFrame(final_weights[max_sharpe_indexId], index = portfolio_stocks_df.columns, columns = ['Weights_Allocation'])
#Rounding the values to 2 digits
max_sharpe_allocation.Weights_Allocation = [round(i*100, 2) for i in max_sharpe_allocation.Weights_Allocation]
#Transposing Stock names from index to columns
max_sharpe_allocation = max_sharpe_allocation.T
#Step 3
min_vol_indexId = np.argmin(final_results[1])
portfolio_min_return, portfolio_min_volatility = final_results[0, min_vol_indexId], final_results[1, min_vol_indexId]
min_vol_allocation = pd.DataFrame(final_weights[min_vol_indexId], index = portfolio_stocks_df.columns, columns = ['Allocation'])
min_vol_allocation.Allocation = [round(i*100, 2) for i in min_vol_allocation.Allocation]
min_vol_allocation = min_vol_allocation.T
#Step 4
print("_" * 125)
print("Maximum Sharpe Allocation \n")
print("Annual Portfolio Return", round(portfolio_return, 4))
print("Annual Risk / Volatility", round(portfolio_volatility, 4))
print("Maximum Sharpe Ratio", final_results[2, max_sharpe_indexId])
print("\n")
print(max_sharpe_allocation)
print("_" * 125)
print("Minimum Volatility Annual Portfolio Return")
print("Annual Return", round(portfolio_min_return, 2))
print("Annual Risk / Volatility", round(portfolio_min_volatility, 2))
print("Sharpe Ratio", final_results[2, min_vol_indexId])
print("\n")
print(min_vol_allocation)
#Step 5 - Visualization
plt.figure(figsize=(20, 12))
plt.scatter(final_results[1, :], final_results[0, :], c = final_results[2, :], cmap = 'YlGnBu', s = 10, alpha = 3)
plt.colorbar()
plt.scatter(portfolio_volatility, portfolio_return, marker = '*', color = 'r', s = 500, label = 'Maximum Sharpe Ratio')
plt.scatter(portfolio_min_volatility, portfolio_min_return, marker = '*', color = 'g', s = 500, label = 'Minimum Volatility')
plt.title("Simulation of Portfolio Optimization based on Efficient Frontier")
plt.xlabel("Annualised Volatility / Risk")
plt.ylabel("Annualised Returns")
plt.legend()
In [51]:
efficient_frontier_random_portfolios(num_assets, num_portfolios, mean_returns, cov_matrix, risk_free_rate)
- The Green star indicates the Low risk strategy portfolio which implies for the minimum risk the the return is 16%
- The Red star indicates the Max Sharpe Ratio - High risk strategy where for the maximum risk the optimial portfolio return is around 38%
4.2. Using 'PYOPT' library for Linear Optimization¶
In [28]:
from pypfopt.efficient_frontier import EfficientFrontier
from pypfopt import risk_models
from pypfopt import expected_returns
This library uses the sharpe ratio as the objective function and optimises the weights as per the investment strategy
In [29]:
# Calculate expected returns and sample covariance
mu = expected_returns.mean_historical_return(portfolio_stocks_df) #Mean Returns
S = risk_models.sample_cov(portfolio_stocks_df) # Annualised covariance matrix
4.2.1 Strategy 1 -> High RIsk - High Return (Max Sharpe Ratio)¶
In [30]:
# Calculate expected returns and sample covariance
Mean_returns = expected_returns.mean_historical_return(portfolio_stocks_df) #Mean Returns
Cov_matrix = risk_models.sample_cov(portfolio_stocks_df) # Annualised covariance matrix
# Optimise for maximal Sharpe ratio
#The range for weights is given between '0' and '1' so that we don't have any negative allocation.
ef = EfficientFrontier(Mean_returns, Cov_matrix, weight_bounds=(0,1))
raw_weights = ef.max_sharpe()
cleaned_weights = ef.clean_weights()
print(cleaned_weights)
ef.portfolio_performance(verbose=True)
Out[30]:
Creating a dataframe for the Max Sharpe Ratio weights¶
In [31]:
cleaned_weights_MS= pd.DataFrame(list(cleaned_weights.items()), columns= ['Symbol', 'OPT_Max_Sharpe_Weight'])
#Rounding the weights
cleaned_weights_MS['OPT_Max_Sharpe_Weight'] = np.round(cleaned_weights_MS['OPT_Max_Sharpe_Weight'] * 100, 2)
#Merging with main dataframe for other stock details
cleaned_weights_MS = pd.merge(cleaned_weights_MS, portfolio_stocks_list, how = 'left')
#And finally sorting with "Category" column for better visualization analysis
cleaned_weights_MS = cleaned_weights_MS.sort_values('Category')
4.2.2. Strategy 2 -> Low Risk - Low Return¶
In [32]:
# Optimise for maximal Sharpe ratio
ef = EfficientFrontier(mu, S, weight_bounds=(0,1))
# The input for finding requried low risk are Expected risk and Risk Free Rate
# Here I've given the acceptable risk to 8% and risk free rate '6.9%'
raw_weights = ef.efficient_risk(0.08, 0.069)
cleaned_weights_low_risk = ef.clean_weights()
print(cleaned_weights_low_risk)
ef.portfolio_performance(verbose=True)
Out[32]:
Although we have given the minimum risk acceptance level at 8% but with this current seletion of stocks 14.4% is the least risk possible.
In [33]:
cleaned_weights_low_risk= pd.DataFrame(list(cleaned_weights_low_risk.items()), columns= ['Symbol', 'OPT_Low_Risk_Weight'])
#Rounding the weights
cleaned_weights_low_risk['OPT_Low_Risk_Weight'] = np.round(cleaned_weights_low_risk['OPT_Low_Risk_Weight'] * 100, 2)
#Merging with main dataframe for other stock details
cleaned_weights_low_risk = pd.merge(cleaned_weights_low_risk, portfolio_stocks_list, how = 'left')
#And finally sorting with "Category" column for better visualization analysis
cleaned_weights_low_risk = cleaned_weights_low_risk.sort_values('Category')
4.2.3. Visualizing the weights for Strategies¶
In [34]:
fig1 = px.bar(cleaned_weights_MS, x = 'Symbol', y = 'OPT_Max_Sharpe_Weight',
color= 'Category',
title = 'High Risk - High Return Strategy (Max Sharpe Ratio)',
labels = {'Symbol': 'Stock', 'OPT_Max_Sharpe_Weight': 'Max Sharpe Weight'})
fig1.show()
fig2 = px.bar(cleaned_weights_low_risk, x = 'Symbol', y = 'OPT_Low_Risk_Weight',
color= 'Category',
title = 'Low Risk Strategy',
labels = {'Symbol': 'Stock', 'OPT_Low_Risk_Weight': 'Low Risk Weight'})
fig2.show()
- As seen from the visualisations earlier on the volatility of the stocks we assumed that for high risk more weight could be given to high volatile stocks.
- Clearly for high risk strategy the algorithm has given '0' weight for Bluechip stocks and moderate weights for midcap and high weight for small cap stocks.
- Similarly for a low risk strategy more than 50% weightage is given to Bluechip stocks.